Ah, so I'm not the only one this happens to
I just take my trusty dull x-acto knife and run it down each groove, starting from the handle and going toward the tip. Usually lifts all the compacted wood chips out with minimal effort. But do be careful not to rub it against the actual cutting edges, so as not to dull them.You can also use a pointy piece of wood, but push from tip to handle so you're not compressing the dust into a tighter space, since it doesn't get up underneath the wood chips as well as the knife. No danger of dulling the reamer, but also doesn't work as well, especially when the wood chips are tightly compacted.
IME, you don't have to clean it out every 4 turns. Just once or twice per peg. But it's still kind of a pain.
I'm glad I don't own any spiral reamers.